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It is a relatively new idea in higher education that students be part 
of the process in evaluating the effectiveness and professional role perform- 
mace of college and university faculty members as classroom instructors. 
Although no one has ever doubted that students have opinions about the quality 
of instruction they receive, it is only uithin the past few decades that attempts 
have been made to systematically gather these judgments. Numerous authors 
(Bornheimer, Burns, & Dumke, 1973; Keast & Macy, 1973; and Miller, 197C'; have 
argued for the inclusion of student ^^nput into the processes of faculty evalu- 
ation and personnel decisions. Others (Centra, 1972; Costin, Greenough, & 
Menges, 1971; Menges, 1973) have argued that ratings can provide a valuable 
resource and incentive for instructors to improve their teaching effectiveness. 
Af\ a consequence, there has been a sharp increase in efforts to utilize student 
ratings of instruction in many colleges and universities for a variety of pur- 
poses. 

However, because any evaluation of an instructor is an implied conparison 
with other instructors whom the student rater has experienced, a direct quanti- 
tative comparison among instructors would seem to be valid only if those being 
evaluated are f*^.;^ a similar or homogeneous population of faculty members. 
Thus, comparative results among various types of instructors Tr:rould seem recessary 
in order to interpret more adeqmtely the eif^^irical meaning of the ratings. 
However, such an interpretation involves the additional difficulty of ascertain- 
ing the extent to which student ratings independently describe teaching profici- 
ency, and the extent to which they are related to certain attributes of the 
student rater, other dimensions of the course, and/or aspects of the faculty 
member's role apart from those directly rated by the student* 

This study sought to contribute to this research by examining the rela- 
tionship between various »^.adent, instructor ^ and course characteristics, and 
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student ratings of instruction. Although there have been numerous studies of 
the correlates of student ratings of instruction^ comparatively few efforts 
have specifically focused on the impact of the combined association of these 
various cha:'v*.cteristics on the ratings. 

Because student ratings are increasingly available, correlational studies 
testing these possible relationships tend sometimes to find statistically signi- 
ficant differences 5 due primarily perhaps to the rather large sample sizes tested. 
Little has been done to describe the practical importance of such findings in 
applying student ratings to the variety of purposes for wh'lch they are commonly 
usedo Moreover, caution must also be taken in making causal inferences between 
various background characteristics and ratings. This investigation sought to 
contribute some perspective to these issues. 

Related Literature 

Student Characteristics 

There is considerable research reported in the literature which has 
attempted to test the possible relationships between student ratings and various 
background and course-related characteristics of the student raters, such as 
their sex, major, year in college, motivation for taking the course, expected 
grade and actual grade in the course, etc. After an extensive review of studies 
testing these variables, Kulik and McKeachie (1975) concluded that correlations 
between ratings and such variables are seldom higfji, even when they are signifi- 
cantc Hayder's (1968) multiple regression stuc^ of ii,285 students' ratings of 
87 ii- actors at Colorado State College found that, when the student character- 
istics of age, sex, grade level, and major were used collectively to predict 
ratings, less than two percent of the variance in ratings were predictedo A more 
recent study of 1,200 courses (Rosenshine, Cohen & Furst, 1973) similarly found 
no relationship of practical significance between student ratings and the student 
variables of age, sex, grade point average, marital status, year in school, and 
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number of previous courses in the fieldo Nc>n0*^^less, these investigations 
did find a weak tendency for students expeC^irJ^ higher grades to gi^ higher 
ratings o 

A number of other studies found no r^la^^^nship tetween the ratings of 
indivivJial students and their expected or aOti^^^ grades in a course (Bendig, 
1953j Eckert, 19^0; Garverick & Carter, 1962; ^J^^^hrie, 19^9; Heilman & Armentrout, 
1936; Hudelson, 1951; Remmers^ 196O; Voeks & J^^^oh, 196O). conti^ast, a 
substantial number of investigators have foi^id ^^gnif icaii't 5 positive relation- 
ships between students' grades and their rB^ir^^ of instructors and coTiraes 
(Caffrey, 1969; Elliott, 1950; Rayder, 1968; Bi^^^^ll & Bendig, 1953; Stewart & 
Malpass, 1966)0 However, these relationships ^^^Q typically weak (r <.30), 

Instructor Characteristics 

Numerous investigators have correlai>^^ ^^oific teacher characteristics 
with student ratings, Costin, Greenough, qlM I^^^ges (l971) have re^^wed 
research on teaching experience and student ^a*^^s, and concluded *hat the 
results are fairly consistent. Several stxidXe^ ^Centra^ 1976 j Claris & Keller, 
195iA; Downie, 1952; Guthrie, 19ii9; Walker, toxmd that student ratings 

tended to improve with the experience of tb^ iJ^^^^ctor** 

With regard to a closely related vari^bJ^^ that of raiilc. Gage (196I) 
found that associate professors and full prc^e^^^^s recei^d significantly higher 
ratings than did instructors or assistart pr^^f^^^^rs. At the Color^^^Q State 
College, Heilman and Armentrout (1936) found nO Significant relationship between 
the years of teaching experience of k6 teach^r^ ^^d their ratings (ii=2,2l5) re- 
ceived from students; while Rayder (1968), 3't tl^^ same institution ^^^^e 30 years 
later, found negative correlations between s^ud^^t ratings (n=I|,285) the 
teaching e^erience of 8? instructors, Simi^^a^^y* at Brooklyjx College, Riley, 
Hjran, and Lifshitz (1950) found a predominately ^^gative relationship between 
Q academic rank and student ratings* While jUi^ o^^^^ll rel^'^^ionship between 
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student ratings and teaching experience and rank -^s j obab^y positive, the size 
and direction of the relationship may vary somew^^ at d3"^^*ei*ent types of insti- 
tutions. 

Other instructor characteristics, such as age and sex, seem little related 
to student ratings (Costin, Greenough & Menges, 1971; Do^e^ 19^2; Elliott, 1950} 
Heilman & Arraentrout^ 1936; McKeachie, 1973) « Hcfwever, B^yder (1968) foxind that 
multiple regression equations, including such instructor characteri^^tics as sex, 
age, raiik, degree and department, predicted mean student ^^'^tingg with greater 
success than equations based on student charactexlstics* ^uationS with instruc- 
tor characteristics predicted between 1^% and 27% of the ^ria^ce x^iean student 
ratings while equations with student characteristics predicted XesB than 2^!^ of 
the variance. 
Course Characteristics 

In addition to the characteristics of the instr'uctor^ the conditions under 
which he teaches may make a difference in the ratings received fron^ students. 
One such variable, that of class size or the number of students ertc^^^^^ ±n the 
course^ has been shown to be negatively related to student rati^^gg in several 
studies (Centra, 1976; Heilman & Armentrout, 1936; Loirell & Han^r, 19^^! Mc Daniel 
& Feldhusen, 1970). However, a few investigators (GoocJb^t^^ 19^8 J Guthrie, 1951;; 
Jiobu & Pollis, 1971) failed to find a relationship between rating^ class 
enroliment. Gage (1961) reported a curvilinear relation^^^P in whi^^ courses 
with enrollments of 30 to 39 undergraduates at the University of Illinois received 
lower ratings than did courses with either mors or f©ver ^Ments. However, 
further research is needed to substantiate the extent of this relationship^ 

Little research has been reported which has tested the contribution to 
rating differences of department and subject areas of the ooiirseo Based on a 
iTUltiple regression stuc3t7, Rayder (1968) concluded that art coxoc^qqs ^7 receive 
higher ratings than courses in mathematics • However, Coben and Huit^^^^^ys (1960) 
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found that psychology majors tended to rate psychology courses and instructors 
about the same as non-psychology majors> although students required to take a 
ps:;^chology course tended to rate it lower than did students who selected the 
coiirse as an elective. Thus, apart from the enrollment size of the course which 
still remains an area of some controversy, there is little consensus in the 
available research t-^ demonstrate that characteristics of the course hava any 
demonstrated influence on the ratings given by students* 

Method 

Sample 

Student rating data were ccMected from undergraduates in libei^l arts 
and professional education coiirses toward the end of each of the four semesterd 
of the 1973-7i4 and 197i4"75 academic years in six colleges of a large urban ijni- 
versityo Mainly introductory courses distributed widely across the htmianities 
and physical, social, and behavioral sciences , all the classes Included students 
from a number of departments and usually from more than one school o^ collegs 
of the university* 
Instrumentation 

The rating instrument used in this study was the Student Instru c tional 
Report ( sir ), a 39-item standardized rating form developed by Centra (1973)* 
Factor analysis of the SIR produced six dimensions: Teacher-Student RelationshipL^, 
Course Objectives and Organization, Quality of Lectures, Quality of Reading Assign- 
ments, Course Difficulty/Workload, and Examinations (Centra, 1973). Because they 
include the most items and contain the t^ro overall summary rating items, this 
stuctjr used only the first two dimensions as dependent variables: (l) Teachei*- 
Student Relationship measures the degree to which an instructor is open to student 
viewpoints, is concerned about their learning, etc. (8 items); and (2) Course 
Obje tives and Organization assesses how well the instructor organized the 
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course and has achieved his objectives, (7 items). Seven descriptive items on 
the SIR assessed relevant student characteristics and course related data such 
as the student's sex, class level, expected grade for the course, motivation 
for taking the course etc. In addition a coversheet completed by the course 
instructor at the time that the course was rated by students, provided additional 
data regarding instructor and course-related characteristics which enabled the 
individual studeni, rating sheets to be aggregated into coui-se units. More speci- 
fically, it contained several questions about the instructor such as his rank, 
teaching load, years of teaching experience, and course-related characteristics 
(e^, class enrollment for the course, subject area of the course, general format 
in which the class was conducted, etc.). 

Centra (1973) reported two tests of reliability for the SIR evaluation form, 
both of which suggest sufficient stc.bility of the ratings derived from this 
instrument. In addition, several validity studies of data generated by the SIR 
have been reported (Centra, 1973; Centra & Creech, 1976). 
Procedures 

Correlational analys&s were used to examine some commonly held ideas about 
influences on student ratings, and zero-order correlations were measured with 
Pearson product moment correlation coefficients. TMs preliminaiy analysis served 
to shed light on variables which seeip.ed to be most associated with the ratings„ 
These independent variables were initially suggested from the review of the lit- 
erature reported aboveo Variables on the SIR evaluation form and coversheet which 
were continuous and consequently appropriate for correlational analyses were 
correlated with the two dependent variables. The only exception to this was the 
variable of course subject, which first had to be clustered into fields of similar 

major content and each of these made dichotomous. 

The second technique used in analysos of these data was multiple regression 

which made it possible to assess the relationship between multiple independent 
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variables of major interest and each Ox the dependent variables, student rating 
factor scale scores, as well as to determine the amount of variance accounted 
for by all of the independent variables combinedo Two levels of analysis were 
employed, the individual student rating scores and course mean scores, necessita- 
ting the generation rf two distinct data setso The former linked the instructor 
and course measures onto each individual student record, and the latter used the 
course mean scores for the student's expected grade ratings and other student 
measures o Thus, Ul560 individual cases and 3558 course records were usable units o 
Data Analysis 

Tables 1 and 2 present matrices of correlation coefficients for those 
variables found to be significantly associated with either of the two rating 
factor scales for both individual rating scores and course mean scores • Examin- 
ation of these matrices revealed that the correlations between the background 
characteristics and the rating variables tended to be somewhat higher when de- 
rived from course mean data than were those based on individual student ratings o 
Moreover, the inter- correlations between the independent variables themselves 
tended to remain constant or even slightly decrease • 

Among the variables whose correlation coefficients with both SIR factors 
were nonsignificant and/or very low were the following: student cumulative grade- 
point average, class level, student's sex, and recency of the instructor's re- 
vision in teaching methods. Among those variables which were related to ratings, 
only instructor's rank and years of teaching experience were highly intercorrelated 
(r»»7l). Because the literature has frequently reported correlates between ratings 
and both these variables, they were both included in the regression equations© 

Multiple regression analyses on both the individual student rating scores 
and the course mean scores were performed separately for each of the two SIR factor 
scales, (1) Tea Cher -Student Relationship and (2) Course Objectives and Organization, 
on the following continuous variables entered simultaneously: student expected 
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grade, instructor's rank, teaching experience, teaching course load, class en- 
rollment size, and the course's siibject area. 
Findings and Discussion 

Tables 3 and h present the findings of multiple regressions of the indiv- 
idual student rating scores o The multiple correlation squared (R^) derived for 
both analyses indicates the proportion or percentage of variation in the depend- 
ent variable which can be explained by the independent variables taken collectively 
(Blalock, i960, po3l;6). Consequently, these findings suggest that this combination 
of independent variables account for only 6.7% and 1;% of the variance in the 
ratings for each of the two SIR factor scales respectively. 

In attempting to assess the relative combination of several student, in- 
stnictor and course measures with the variation in the rating scores, examination 
of the individual beta weights (^) were particularly enlightening. Table 3 shows 
that when comparing in standardized Tinits, student expected grade accounted for 
by far the largest proportion of variation in the ratings of Teacher-Student 
Relationship among the independent variables under consideration. Sim3.1arly, 
Table U indicates that student expected grade also accounted for the largest 
proportion of variance in the ratings of Course Objectives and Organization*. 
The beta weight was greatest for student expected grade, while the next highest 
weights were biology, psychology & health science course subject and instructor's 
teaching experience. 

However, while expected grade had by far the greatest predictive value of 
all of the independent variables considered, the size of this value was relatively 
small. Moreover, the findings seem to suggest that class size and rank were 
predictive of somewhat lower ratings of Teacher-Student Relationship, whereas 
Fyediter teaching experience was predictive of slightly higher ratings of both 
Ic'cicher- Student Relationship and Course Objectives and Organization. 

Comparable regression analyses using only course mean scores of the two 
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SIR factor scales, as well as class mean expected grade, are reported in Tables 
5 and 6. These findings indicate that the multiple correlation squared (R^) for 
both SIR factors increases somewhat over those yielded from the individxial student 
rating scores reported above in Tables 3 and ii. Thus, the amount of variance 
accounted for by the combination of these independent variables appears to be 
somewhat exacerbated when the course mean ratings were taken as the unit of analysis 

The beta weights of the mean expected grade and other instructor and coxirse 
variables did not seem to be substantially effected by the shift in the level of 
analysis. However, it is noteworthy to observe that the beta weights for the more 
codified fields, such as biology, psychology, health science, mathematics, physical 
science and business, seemed to increase in negative values, predicting lower 
ratings, especially on the Teacher- Student Relationship factoi- scale. This is 
consistent with an inteipretation made by Wilson et al. (1975) whose findings 
suggested that the moro codified the knc^ledge of a field, the more likely pro- 
fessors were to adopt a tightly focused and structxired teaching style. These 
professors tended to involve students less in the conduct of the class, and their 
classes provided less opportunity for students to participate in the structxire 
of the course content. These findings also argue for the use of course subject 
normative data when comparisons are attempted across academic units. 

In addition, it is also curious to observe that a:'.though rank and teaching 
experience were highly inter-correlated, instructor's irank was again predictive 
of slightly lower ratings, especially on Teacher-StudeiTt Relationship, while 
teaching experience seemed to predict somewhat higher ratings, most especially 
on Course Objectives and Organization. This may suggest that junior faculty mem- 
bers my be more inclined toward student-centered teaching practices than their 
senior colleagues, but that course planning and organization tends to improve 
somewhat with an increase in teaching experience. 

In conclusion, these findings seem to suggest that students' expected grade 
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in a course does -liccount for the largest amount of variance in student ratings 
when compared wibh the relative impact of a variety of other instructor and course 
related variables. However, even when combined, those variables were fo\ind to 
account for comparatively little variability in student ratings. Consequently, 
the biases contributed by such factors seemed to be quite small and did not seem 
likely to greatly effect the ratings. These findings are encourageing because 
they support the validity of student ratings as measures of instructor performance,. 

Nonetheless, future research efforts should investigate further the apparent 
exacerbated relationships between student ratings and various instructor and 
course variables when class mean scores are used as the unit of analysis p Further- 
more, these findings suggest that more theoretically-based research on the neaning 
of students' expected grade and its association with their ratings of instruction 
is required before definitive statements can be made about the validity of student 
ratings of instruction, until now only partially addressed in the literature. 
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